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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

Applicant(s) Uirich BENZLER and Oliver WERNER 

Serial No. To Be Assigned 

Filed Herewith 

For METHOD FOR GENERATING AN IMPROVED ^ 

IMAGE SIGNAL WHEN ESTIMATING THE MOTION 
OF IMAGE SEQUENCES, IN PARTICULAR A 
PREDICTION SIGNAL FOR VIDEO IMAGES 
USING MOTION-COMPENSATING PREDICTION 

Examiner To Be Assigned 

Group Art Unit To Be Assigned 

Assistant Commissioner for Patents 
Washington, DC 20231 

PRELIMINARY AMENDMENT 

Sir: 

Kindly amend the above-identified application before examination 

as follows: 

IN THE SPECIFICATION : 

On page 1, delete lines 1 and 2, and insert: 

- - Field Of The Invention- -. 

On page 1, before line 12, insert: 

- - Background Information - -. 

On page 1 , line 12, change "EP 0 558 922 B2" to - -No. 0 558 

922- -. 

On page 1 , line 21 , delete "the". 

On page 2, delete lines 23 and 24, and insert: 



- - Summary Of The Invention- -. 

On page 2, line 25, change "measures" to - -principles- -. 

On page 3, line 7, change "EP 0 558 922 B1" to 

- -No. 0 558 922- -, and before "design" insert - -to- -. 

On page 3, line 12, change "measures" to - -principles- - 

On page 3, delete lines 18-31 and insert: 

- - Brief Description Of The Drawings 
Figure 1 shows a positional diagram. 

Figure 2 shows an interpolation diagram for pixels between the pixel scanning 
raster. 

Figure 3 shows the structure of an FIR filter for interpolation. 

Figure 4 shows a further interpolation having an even higher resolution. 

Detailed Description - -. 

On page 4, lines 14 and 15, change "EP 0 368 151 B1" to - -No. 0 

368 151--. 

On page 8, line 1 , change "Patent Claims" to 

- - What Is Claimed Is - -. 

IN THE CLAIMS : 

Please cancel original claims 1-5, and please cancel substitute 
claim 1 , without prejudice. 
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Please add the following new claims: 

6. (New) A method for generating an image signal when estimating a motion of 
image sequences, motion vectors indicating, for each picture block of a current 
image, a position of the picture block used for a prediction with respect to a 
chronologically preceding reference image, the motion vectors being formed for 
each picture block, the method comprising the steps of: 

in a first search step, determining a first motion vector with a pel 

accuracy; 

starting out from the first motion vector, in a second search step, 
determining a second motion vector with a sub-pel accuracy by an aliasing- 
reducing interpolation filtering, using a digital filter, a resolution being selected to 
be higher than that corresponding to a resolution of a pixel raster in the first 
search step, more than four neighboring pixels being utilized for an interpolation 
of each pixel, to interpolate pixels between a scanning raster for the first search 
step; and 

in a third search step, starting from the second motion vector, 
determining a third motion vector by a further interpolation filtering using the 
digital filter, a resolution being increased once more in comparison with the 
second search step, an interpolation being carried out on the basis of a pixel 
raster, with a resolution in the second search step. 

7. (New) The method according to claim 6, wherein the image signal is a 
prediction signal for video images generated using a motion-compensating 
prediction. 

8. (New) The method according to claim 6, wherein the more than four 
neighboring pixels are more neighboring pixels than are utilized for a bilinear 
interpolation. 

9. (New) The method according to claim 6, wherein, for the interpolation filtering 
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in the second search step, an FIR filter is used having filter coefficients C01 = 
161/256, C02 = -43/256, C03 = 23/256, C04 = -8/256. 

1 0. (New) The method according to claim 6, wherein for the further interpolation 
filtering in the third search step, an FIR filter is used having FIR filter coefficients 
C01' = 1/2, C02' = O, C03' = O, C04' = O. 

1 1 . (New) The method according to claim 6, further comprising the steps of: 

in order to predict video objects, separately conditioning, for each 
video object, filter coefficients of the digital filter; and 

inserting the filter coefficients into a transmission bit stream at a 
beginning of transmission of an object in question. 

12. (New) The method according to claim 6, further comprising the step of: 

adapting, for an encoding of a motion vector for a transmission, a 
range of values of motion vector differences to be coded to an increased 
resolution. 

IN THE ABSTRACT : 

On page 1 0, line 1 , change "Abstract" to 
Abstract Of The Disclosure —. 

On page 10, lines 9-11, change "interpolation methods. 
One" to - -interpolation methods. One- -. 

REMARKS 

This Preliminary Amendment cancels, without prejudice, original 
claims 1-5 and substitute claim 1 in the underlying PCT Application No. 
PCT/DE98/01938, and adds new claims 6-12. The new claims conform the 
claims to U.S. Patent and Trademark Office rules and do not add new matter to 
the application. 
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The amendments to the specification and abstract are to conform 
the specification and abstract to U.S. Patent and Trademark Office rules, and do 
not introduce new matter into the application. 

The underlying PCT Application No. PCT/DE98/01938 includes 
an International Search Report, dated December 16, 1998, a copy of which is 
included. The Search Report includes a list of documents that were considered 
by the Examiner in the underlying PCT application. 

The underlying PCT Application No. PCT/DE98/01938 also 
includes an International Preliminary Examination Report, dated June 17, 1999, 
a copy of which is included, including a translation. 

Applicants assert that the present invention is new, non-obvious, 
and useful. Prompt consideration and allowance of the claims are respectfully 
requested. 



Respectfully Submitted, 



KENYON & KENYON 





Richard L. Mayer 
Reg. No. 22,490 



One Broadway 
New York, NY 10004 
(212) 425-7200 
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METHOD FOR GENERATING AN IMPROVED IMAGE SIGNAL WHEN 
ESTIMATING THE MOTION OF IMAGE SEQUENCES, IN PARTICULAR A 
PREDICTION SIGNAL FOR VIDEO IMAGES USING MOTION-COMPENSATING 
PREDICTION 

Background Information 

The present invention relates to a method for generating an 
improved image signal when estimating the motion of image 
sequences, in particular a prediction signal for video 
images using motion-compensating prediction, with motion 
vectors, which, for each picture block of a current image, 
indicate the position of the picture block used for the 
prediction with respect to a chronologically preceding 
reference image, being formed for picture blocks. 

European Patent EP 0 558 922 B2 describes a method for 
improving motion estimation in image sequences, in half-pel 
accuracy, according to the full-search method. There, in a 
first process step, the search area, and in a second process 
step, the match block are filtered with the aid of an 
additional digital filter which enables a raster shift of 
the pixel raster by H pel. Using this measure, a distortion 
of the motion vector field can be ruled out. 

In the *MPEG-4 Video Verification Model Version 7.0", 
Bristol, April 1997, MPEG 97/N1642 in ISO/IEC JTC1/SC 
29/WG11, an encoder and decoder for object-based coding of 
video image sequences are specified. In this context, one no 
longer encodes and transmits rectangular pictures of a fixed 
size to the receiver, but instead, so-called * VIDEO OBJECTS" 
(VO) of any shape and size. The image formation of such a VO 
in the camera image plane at a specific instant is referred 
to as a VIDEO OBJECTS PLANE (VOP) . Consequently, the 



relation between VO and VOP is equivalent to the relation 
between image sequence and image in the case of the 
transmission of rectangular pictures of fixed size. 

5 The motion-compensating prediction in the verification model 
is carried out with the assistance of so-called "blockwise 
motion vectors" which, for each block of the size 8 x 8 or 
16 x 16 pixels of the current image, specifies the position 
of the block used for the prediction in an already 
10 transmitted reference image. In this context, the resolution 
of the motion vectors is limited to half of a pixel, pixels 
between the scanning raster (half-pixel position) being 
generated by a bilinear interpolation filtering from the 
,7; pixels on the scanning raster (integer pixel position) 
ipB (Figure 1) . In this case, + indicates the integer-pixel 

pi position, and 0 the half-pixel position. Interpolated values 

a, b, c, and d in half-pixel position are derived from the 
yj following relations: 

1 a = A, b = (A + B)//2, c = (A + C)//2, 

it) d = (A + B + C + D)//4, // indicating a rounded integer 

D 

fp division. 

D 

Summary of the Invention 

25 By applying the measures of the present invention, one can 
improve the quality of the prediction signal and, thus, the 
coding efficiency. In so doing, a greater local neighborhood 
is considered than in the case of bilinear interpolation, to 
generate pixels between the pixel scanning raster. The 

30 aliasing-reducing interpolation filtering according to the 

present invention leads to an increased resolution of the 
motion vector and, consequently, to a prediction gain and an 
increased coding efficiency. In the present invention, the 
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FIR filter coefficients can be adapted to the signals to be 
coded, and be transmitted separately for each video object, 
thereby further increasing coding efficiency and enhancing 
the flexibility of the method. 

In contrast to the design approach according to European 
Patent EP 0 558 922 Bl, there is no need design any 
additional polyphase filter structures for intermediate 
positions having H pel pixel resolution in the horizontal 
and vertical directions. 

By applying the measures of the present invention, the image 
sequence frequency of an MPEG-1 coder can be doubled from 25 
Hz to 50 Hz, with the data rate remaining constant. In the 
case of an MPEG-2 coder, the data rate can be reduced by up 
to 30%, with the image quality remaining constant. 

Brief Description of the Drawings 

Exemplary embodiments of the present invention are explained 
in the following in greater detail, on the basis of the 
drawing, whose figures show: 

Figure 2 an interpolation diagram for pixels between the 

pixel scanning raster; 
Figure 3 the structure of an FIR filter for interpolation; 
Figure 4 a further interpolation having an even higher 

resolution. 

Description of Exemplary Embodiments 

In the case of the method according to the present 
invention, motion vectors are formed for picture blocks, the 



motion vectors, for each picture block of a current image, 
indicating the position of the picture block used for the 
prediction with respect to a chronologically preceding 
reference image. 

The motion vectors for the prediction are determined in 
three successive steps: 

In a first search step, a motion vector is determined for 
each picture block with pel accuracy in accordance with a 
conventional method, for example, in accordance with the 
full-search block matching method. In this context, the 
minimum error criterion is determined for possible motion 
positions, and the vector which best describes the motion of 
the picture block is selected (European Patent EP 0 368 151 
Bl) . 

In a second search step, which, again, is based on such a 
search for the minimum error criterion, an improved motion 
vector is ascertained with sub-pel accuracy, starting out 
from the motion vector ascertained in the first step, using 
an aliasing-reducing interpolation filtering, with the aid 
of a digital, symmetric FIR (finite impulse response) 
filter. In the process, a higher resolution is selected than 
in the first search step. Preferably, one selects a 
resolution of a half pixel relative to the pixel raster. 
Figure 2 illustrates the interpolation pattern for pixels b, 
c, and d between the pixel raster, as derived from the 
neighboring pixels A, B, C, D, E, F, G, H on the pixel 
raster. + indicates the integer-pixel position, 0 the half- 
pixel position, so that: 



b = (C01x(A_ 1 + A +1 ) + C02x (A_ 2 + A +2 ) + C03x (A_ 3 + A +3 ) + 
C04x (A_ 4 + A+ 4 ) ) /256 



c x = (C01x{A L + E ± ) + C02x(B ± + FJ + C03x (Q i + G ± ) + G04x (Dj. 
+ HJ ) /256 

d = (COlxic.i + c +a ) + C02x(c_ 2 + c +2 ) + C03x (c_ 3 + c +3 ) + 
C04x (c_ 4 + c +4 ) ) /256 

The structure of the FIR interpolation filter used is 
apparent in Figure 3. Following each pixel position 5p, it 
branches to a coefficient evaluator 1, 2, 3, etc., and has a 
summing device 10 at the output. As is apparent from the 
above relationships, a greater local neighborhood is 
considered when generating pixels between the scanning 
raster, than in the case of the bilinear interpolation 
according to the related art. In this context, the 
interpolation filter coefficients C02, C03, C04 are 
determined in a way that minimizes the interpolation error 
performance. The coefficients can be determined directly 
using the known estimation method of the least root-mean- 
square error. From the minimization of the interpolation 
error performance, one obtains a linear system of equations, 
whose coefficients can be derived from the principle of 
orthogonality. A set of FIR filter coefficients optimized in 
this manner is given by the coefficients COl = 161/256, C02 
= -43/256, C03 = 23/256, C04 = -8/256. 

In the third search step, starting from the motion vector 
determined with an accuracy of H pel, a local search is 
performed using a further interpolation filtering, taking 
the eight neighboring pixels as a basis, with resolution 
that is increased still further, preferably to H pixel. As 
before, one selects the motion vector having the lowest 
prediction error performance. 



Figure 4 shows the interpolation pattern for this. The 



integer pixel positions are marked by.X, the half-pixel 
positions by 0, and the quarter-pixel positions by -. 0 
indicates the best compensation with H pixel, and + the 
quarter-pixel search position. 

The interpolation is carried out relative to the pixel 
raster, with a half-pixel resolution from the second search 
step, using filter coefficients C01 T = 1/2, C02 ' = O, C03 ' = 
0, C04' = 0. 

The same previously introduced interpolation technique is 
used for the motion-compensating prediction. 

If the processing is carried out within a coder having a 
reduced image format (SIF format within an MPEG1 coder or Q- 
CIF in an H.263 coder), but the original input format is 
used for the display, for example, CCIR 601 [1] in the case 
of MPEG-1 or CIF in the case of H.263, a local interpolation 
filtering must be carried out as a post-processing. The 
described aliasing-compensating interpolation filtering can 
be used for this purpose as well. 

To activate the aliasing-compensating interpolation using 
1/4 resolution, activation bits can be inserted into an 
image-transmission bit stream. 

To predict video objects, filter coefficients C01 through 
C04, and C01 1 through C04 ' can be separately conditioned for 
each of the video objects VO, and inserted into the image- 
transmission bit stream at the beginning of transmission of 
the video object in question. 

For the encoding of a motion vector, the range of values of 



the motion vector differences to be ceded can be adapted to 
the increased resolution. 
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Patent Claims 



1 . A method for generating an improved image signal when 
estimating the motion of image sequences, in particular a 
prediction signal for video images using motion-compensating 
prediction, with motion vectors, which, for each picture 
block of a current image, indicate the position of the 
picture block used for the prediction with respect to a 
chronologically preceding reference image, being formed for 
picture blocks, comprising the following steps: 

- in a first search step, a motion vector is determined with 
pel accuracy; 

- starting out from the motion vector, in a second search 
step, an improved motion vector is ascertained with sub-pel 
accuracy by aliasing-reducing interpolation filtering, using 
a digital filter, the resolution being selected to be higher 
than that corresponding to the resolution of the pixel 
raster in the first search step, and more neighboring pixels 
being utilized for the interpolation than in the case of a 
bilinear interpolation; 

- in a third search step, starting from the motion vector 
determined with sub-pel accuracy, a further improved motion 
vector is determined by a further interpolation filtering 
using the digital filter, the resolution being increased 
once more in comparison with the second search step, and the 
interpolation being carried out on the basis of the pixel 
raster, with the resolution in the second search step. 

2. The method as recited in Claim 1, 

characterized in that, for the interpolation filtering in 
the second search step, an FIR filter is used having the 
filter coefficients C01 = 161/256, C02 = -43/256, C03 = 
23/256, C04 = -8/256. 
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3. The method as recited in Claim 1 or 2,, 
characterized in that, for the interpolation filtering in 
the third search step, an FIR filter is used having the FIR 
filter coefficients C01 ' = 1/2, C02 ' = 0, C03 ' = 0, C04 « = 
0. 

4. The method as recited in one of the Claims 1 through 3, 
characterized in that, to predict video objects (V0) , the 
filter coefficients of the digital filter/FIR filter are 
separately conditioned for each video object, and inserted 
into a transmission bit stream at the beginning of 
transmission of the object in question. 

5. The method as recited in one of the Claims 1 through 4, 
characterized in that, for the encoding of a motion vector, 
in particular for a transmission, the range of values of the 
motion vector differences to be coded is adapted to an 
increased resolution. 
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Abstract 



To generate an improved image signal when estimating motion, 
a motion vector is first determined with pel accuracy, and a 
two-step interpolation filtering is subsequently carried out 
with sub-pel accuracy. The interpolation coefficients are 
selected with a view to reducing aliasing. More neighboring 
pixels are used for the interpolation than in usual 
interpolation methods. 

One is able to improve the quality of the prediction signal 
for video images and, consequently, enhance coding 
efficiency . 
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COMBINED DECLARATION AND 
POWER OF ATTORNEY FOR PATENT APPLICATION 

As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below 
adjacent to my name. 

I believe I am the original, first and sole inventor (if only one name is listed 
below) or an original, first and joint inventor (if plural names are listed below) of the 
subject matter which is claimed and for which a patent is sought on the invention 
entitled "METHOD FOR GENERATING AN IMPROVED IMAGE SIGNAL WHEN 
ESTIMATING THE MOTION OF IMAGE SEQUENCES, IN PARTICULAR A 
PREDICTION SIGNAL FOR VIDEO IMAGES USING MOTION-COMPENSATING 
PREDICTION", and the specification of which: 

[ ] is attached hereto; 

[ ] was filed as United States Application Serial No. on 

, 19 and was amended by the Preliminary 

Amendment filed on , 1 9 . 

[X] was filed as PCT International Application Number 
PCT/DE98/01938, on the 1 1 th day of July, 1998. 
[X] an English translation of which is filed herewith. 
I hereby state that I have reviewed and understand the contents of the 
above-identified specification, including the claims, as amended by any amendment 
referred to above. 

I acknowledge the duty to disclose information which is material to the 
examination of this application in accordance with Title 37, Code of Federal 
Regulations, §1 .56(a). I hereby claim foreign priority benefits under Title 35, United 
States Code § 1 19 of any foreign application(s) for patent or inventor's certificate or of 
any PCT international applications(s) designating at least one country other than the 
United States of America listed below and have also identified below any foreign 
application(s) for patent or inventor's certificate or any PCT international application(s) 
designating at least one country other than the United States of America filed by me on 



the same subject matter having a filing date before that of the application(s) of which 
priority is claimed: 



PRIOR FOREIGN/PCT APPLICATION(S) 

AND ANY PRIORITY CLAIMS UNDER 35 U.S.C. § 119 

Country : Germany 

Application No. : 1 97 30 305.6 

Date of Filing: July 15, 1997 

Priority Claimed 

Under 35 U.S.C. §119: [X] Yes [ ] No 

I hereby claim the benefit under Title 35, United States Code § 120 of any United 

States Application or PCT International Application designating the United States of 

America that is/are listed below and, insofar as the subject matter of each of the claims 

of this application is not disclosed in that/those prior application(s) in the manner 

provided by the first paragraph of Title 35, United States Code § 1 12, I acknowledge 

the duty to disclose material information as defined in Title 37, Code of Federal 

Regulations § 1 .56(a) which occurred between the filing date of the prior application(s) 

and the national or PCT international filing date of this application: 

PRIOR U.S. APPLICATIONS OR 
PCT INTERNATIONAL APPLICATIONS 
DESIGNATING THE U.S. FOR BENEFIT UNDER 35 U.S.C. § 120 

U.S. APPLICATIONS 

Number : 

Filing Date : 

PCT APPLICATIONS 
DESIGNATING THE U.S. 

PCT Number : 

PCT Filing Date : 
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I hereby appoint the following attorney(s) and/or agents to prosecute the 



above-identified application and transact all business in the Patent and Trademark 

Office connected therewith. 

(List name(s) and registration number(s)): 



jDjieJBrosdway ~* 
^lSlew^rk^ew^odclQQ04__ 

Telephone No.: (212) 425-7200 
Facsimile No.: (212) 425-5288 



I hereby declare that all statements made herein of my own knowledge 
are true and that all statements made on information and belief are believed to be true 
and further that these statements were made with the knowledge that willful false 
statements and the like so made are punishable by fine or imprisonment or both under 
Section 1001 of Title 18 of the United States Code and that such willful false statements 
may jeopardize the validity of the application or any patent issuing thereon. 




Richard L. Mayer, 
Gerard A. Messina, 



Reg. No22J90 
Reg. Na_35,9^52 

Reg. No. 

Reg. No. 



All correspondence should be sent to: 



Richard L. Mayer, Esq. 
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Full name of inventor Ulrich BENZLERX 

Inventor's signature fk fyuJIvf Patent) - 

Citizenship Federal Republic of Germany 

Residence Am Krahenberg 18 

30855Xangei±agen_ 

Federal Republic of Germany ^ 

Post Office Address Same as above 
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Full name of inventor ^ ^Ijver WERNER \ 

Inventor's signature U.^-^-4> — S Date At fjr. 2o oo 
Citizenship Federal Republic of Germany 



Residence Flat/Birdhurst Roa^&oQfh Croydon _ 

Surrey CR^E^C lo«-<U»** JkoiJL^ . yU i 

GreatSrifaln v 20 zto UJk^ 



Post Office Address Same as above 
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